A Unified Framework for Correlation Mining in Ultra-High Dimension

Many applications benefit from theory relevant to the identification of variables having large correlations or partial in high dimension. Recently there has been progress ultra-high dimensional setting when sample size $n$ is fixed and dimension notation="LaTeX">$p$ tends infinity. Despite these advances, correlation screening framework suffers practical, methodological theoretical deficiencies. For instance, previous requires that population covariance matrix be sparse block diagonal. This sparsity assumption however restrictive practical applications. As a second example, estimation dependence measures, which can computationally prohibitive. In this paper, we propose unifying approach mining not restricted diagonal structure, thus yielding methodology suitable for modern By making connections random geometric graphs, number highly correlated are shown have compound Poisson finite-sample characterizations, hold both finite case The also demonstrates duality between with consequences.